Measuring linguistic markers and linguistic noise of a machine-human translation supply chain

ABSTRACT

An approach is provided in which a linguistic analyzer engine generates a leverage value of a language translation supply chain that corresponds to an amount of suggested translations that are accepted by a professional linguist. The linguistic analyzer engine generates a factor value of the language translation supply chain that indicates a productivity of the user to convert the set of accepted translation into a set of final translations. In turn, the linguistic analyzer engine determines a performance efficiency of the language translation supply chain based upon the generated leverage value and the generated factor value, and evaluates the language translation supply chain accordingly. In one embodiment, the linguistic analyzer engine determines a performance efficiency of the language translation supply chain based on “n” distinct metric values associated with final translated segments.

BACKGROUND

A company typically develops web pages in a native language (e.g.,English) and subsequently employs a language translation service totranslate the company's web pages into different languages (e.g.,Spanish, Russian, etc.). Language translation services typically utilizea translation supply chain that includes an integration of linguisticassets/corpuses, translation automated systems, computer-aidedtranslation editors, professional linguists, and operational managementsystems. The translation supply chain typically includes a languageasset optimization stage, a machine translation stage, and apost-editing stage.

The language asset optimization stage parses source content into sourcesegments then searches a repository of historical linguistic assets forthe best suggested translations per language and per domain. Linguisticassets may be historical translation memories (bi-lingual segmentdatabases), dictionaries, and/or language specific metadata to optimizedownstream stages. The machine translation stage customizes atranslation model using domain specific linguistic assets of a givenlanguage and provides machine-generated suggested translations oforiginal content based upon the customized translation model.

In turn, the post-editing stage provides the suggested translations to aprofessional linguist via a computer-aided translation editor. Theprofessional linguist accepts one of the suggested matchingtranslations, modifies one of the suggested matching translations, orgenerates a completely new translation and delivers final human fluenttranslated content to the company.

BRIEF SUMMARY

According to one embodiment of the present disclosure, an approach isprovided in which a linguistic analyzer engine generates a leveragevalue of a language translation supply chain that corresponds to anamount of suggested translations that are accepted by a professionallinguist. The linguistic analyzer engine generates a factor value of thelanguage translation supply chain that indicates a productivity of theuser to convert the set of accepted translation into a set of finaltranslations. In turn, the linguistic analyzer engine determines aperformance efficiency of the language translation supply chain basedupon the generated leverage value and the generated factor value, andevaluates the language translation supply chain accordingly. In oneembodiment, the linguistic analyzer engine determines a performanceefficiency of the language translation supply chain based on “n”distinct metric values associated with final translated segments.

The foregoing is a summary and thus contains, by necessity,simplifications, generalizations, and omissions of detail; consequently,those skilled in the art will appreciate that the summary isillustrative only and is not intended to be in any way limiting. Otheraspects, inventive features, and advantages of the present disclosure,as defined solely by the claims, will become apparent in thenon-limiting detailed description set forth below.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The present disclosure may be better understood, and its numerousobjects, features, and advantages made apparent to those skilled in theart by referencing the accompanying drawings, wherein:

FIG. 1 is an exemplary diagram depicting a translation supply chain anda linguistic analyzer engine that determines a performance efficiency ofthe translation supply chain;

FIG. 2 is an exemplary diagram depicting a table showing a translationmatch progression through a translation supply chain;

FIG. 3 is an exemplary diagram depicting an example of a class set eventdata hierarchy;

FIG. 4 is an exemplary flowchart depicting steps taken by a languageasset optimization stage to generate suggested matches for originalsegments partitioned from original content;

FIG. 5 is an exemplary flowchart depicting steps taken by a machinetranslation stage to customize a machine translation model and translateoriginal segments into a different language using the customized machinetranslation model;

FIG. 6 is an exemplary flowchart depicting steps taken by a post-editingstage that provides suggested translation matches to a professionallinguist and stores final translations determined by the professionallinguist;

FIG. 7 an exemplary flowchart depicting steps taken by a linguisticanalyzer engine to analyze translation log data and determine linguisticnoise in a translation supply chain;

FIG. 8 is an exemplary diagram depicting a two-dimensional graphicalrepresentation of a multi-dimensional linguistic noise coordinatesystem;

FIG. 9 is a block diagram of a data processing system in which themethods described herein can be implemented; and

FIG. 10 provides an extension of the information handling systemenvironment shown in FIG. 9 to illustrate that the methods describedherein can be performed on a wide variety of information handlingsystems which operate in a networked environment.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present disclosure has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the disclosure in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the disclosure. Theembodiment was chosen and described in order to best explain theprinciples of the disclosure and the practical application, and toenable others of ordinary skill in the art to understand the disclosurefor various embodiments with various modifications as are suited to theparticular use contemplated.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions. The following detailed description willgenerally follow the summary of the disclosure, as set forth above,further explaining and expanding the definitions of the various aspectsand embodiments of the disclosure as necessary.

FIG. 1 is an exemplary diagram depicting a translation supply chain anda linguistic analyzer engine that determines a performance efficiency ofthe translation supply chain. Translation supply chain 100, is alanguage translation supply chain that includes various stages oftranslating original content written in a first language to translatedcontent written in a second language, such as translating a web pagewritten in English to a web page written in Spanish. The example in FIG.1 shows that translation supply chain 100 includes language assetoptimization stage 120, machine translation stage 130, and post-editingstage 140. As discussed in more detail below, this disclosure provides asystematic approach for measuring linguistic noise within translationsupply chain 100 to optimize components and operational assets/resourcesaccordingly.

Translation supply chain 100 receives shipment store 110 as part of atranslation request. In turn, language asset optimization stage 120retrieves original content 115 from shipment store 110 to commence thetranslation process. In one embodiment, a “shipment” is a translationwork order that includes original content of multiple documents,multiple strings, and/or web pages. Language asset optimization stage120 parses original content 115 into segments via a segmentation processand then compares each original segment against multiple historicaltranslation segments included in linguistic asset store 125 to identifysuggested translations, which are also referred to herein as “matches”(see FIG. 4 and corresponding text for further details). In oneembodiment, during the parsing process, language asset optimizationstage 120 generates multiple segment sizes corresponding to the amountof words in a segment (e.g., small, medium, large). In anotherembodiment, language asset optimization stage 120 classifies suggestedtranslations, and stores corresponding metadata, into one of thefollowing categories:

-   -   Auto Exact (AE) match: Original segment is an exact match of at        least one historical source segment translation associated with        the same document name. In one embodiment, language asset        optimization stage 120 automatically stores auto exact matches        in shipment store 110 without any corrections needed by a        professional linguist via post-editing stage 140;    -   Exact matches (EM): Original segment is an exact match of at        least one historical source segment translation associated with        different documents (e.g., source context). Since the context        (document) of the matching segment is different, a professional        linguist reviews the exact match and either confirms no variance        in the target translation or corrects contextual variance in the        target translation;    -   Fuzzy Match (FM): Original segment is “similar” to at least one        historical source segment translation. As such, the professional        linguist reviews the match and either confirms no variance in        the target language or corrects any contextual variances in the        pre-existing translation; and    -   No Match (NP): Language asset optimization stage 120 does not        locate an AE, EM, FM or other match type for an original        segment.

Language asset optimization stage 120 stores auto exact matches, exactmatches, and fuzzy matches in shipment store 110 to assist and optimizemultiple downstream components and deliver human fluent translations. Inone embodiment, post-editing stage 140 displays the exact matches andfuzzy matches as “suggested matches” because a professional linguist isrequired to accept and correct one of the suggested matches before thesuggested matches are stored as a final translation (see FIG. 6 andcorresponding text for further details).

Machine translation stage 130 customizes a translation model usinghistorical translation information and then uses the customizedtranslation model to generate translation matches for the originalsegments (see FIG. 5 and corresponding text for further details). Thegenerated translation matches are referred to herein as “machinetranslations (MT),” which post-editing stage 140 also considers as“suggested matches” because the professional linguist may accept andcorrect a machine translation before the machine translation is storedas a final human fluent translation.

In one embodiment, machine translation stage 130 generates a machinetranslation for original segments having no matches from language assetoptimization stage 120. In another embodiment, machine translation stage130 generates a machine translation for each original segment having nomatches as well as segments having fuzzy matches. In this embodiment,linguistic analyzer engine 180 compares the efficiency of the machinetranslations generated by machine translation stage 130 with theefficiency of fuzzy matches generated by language asset optimizationstage 120.

Post-editing stage 140 uses a computer-aided translation editor todisplay the suggested translation matches to a professional linguist andspecial translation editing keys to assist the professional linguist togenerate final human-fluent translations, which are stored as translatedcontent 150 in shipment store 110 (see FIG. 6 and corresponding text forfurther details). Post-editing stage 140 also stores quality segmenttranslations 160 in linguistic asset store 125 to improve language assetoptimization stage 120's subsequent translation proficiency.

In addition, post-editing stage 140 tracks usage of the specialtranslation editing keys and stores the tracking information as eventdata 170 in event store 175, which includes log information such as thenumber of suggested matches accepted by the professional linguist andthe amount of time taken by the professional linguist to accept finaltranslations. Linguistic analyzer engine 180 retrieves event data 170and generates linguistic marker data 185 that, in one embodiment,corresponds to a multi-dimensional linguistic noise coordinate system tomeasure noise over a set of events within translation supply chain 100(stored in linguistic analytics store 190, see FIG. 8 and correspondingtext for further details). In one embodiment, linguistic marker data 185may correspond to a two-dimension space consisting of a post-editingfactor value and a post-editing leverage value. The post-editing factorvalue, in one embodiment, is a normalized measurement of professionallinguist productivity during post-editing stage 140 to performtranslations with the aid of a specific match type, or other aidsnormalized against a base productivity performed without aids. Thepost-editing leverage value, in one embodiment, is a measurement ofaccepted matches by the professional linguist during post-editing stage140 (see FIGS. 7, 8, and corresponding text for further details).

Referring to FIG. 8, linguistic markers are compared to PerfectLinguistic Marker (PLM) 835, which includes a post-editing leveragevalue of 1.0 (x-axis) and a post-editing factor value of 0.0 (y-axis).The post-editing leverage value of 1.0 corresponds to a professionallinguist accepting 100% of suggested matches and a post-editing factorvalue of 0.0 corresponds to the professional linguist requiring zeromechanical effort (no changes) to achieve human-fluent translations (seeFIG. 8 and corresponding text for further details). As such, thedistance from a linguistic marker to PLM 835 represents the noise(errors) introduced across the various translation stages of translationsupply chain 100.

Using the linguistic markers corresponding to the set of eventsperformed within translation supply chain 100, linguistic analyzerengine 180 and an operational team determines the translation supplychain noise of translation supply chain 100. In one embodiment, thetranslation supply chain noise represents a singular measurementcomputed by weighting the total number of words translated within thetranslation supply chain by the distance of the corresponding linguisticmarkers to the PLM. In this embodiment, the translation supply chainnoise measurement indicates a cumulative measurement of all noisemeasurements within translation supply chain 100. As such, linguisticanalyzer engine 180 and the operational team adjust translation supplychain 100 accordingly for optimal efficiency.

In one embodiment, linguistic analyzer engine 180 and the operationalteam evaluate the way in which changes in operational assets/resourcesand components affect patterns in translation supply chain linguisticnoise measurements. For example, the operational team may quantifychanges in the supply chain noise as various machine translation serviceproviders are added/deleted in translation supply chain 100 or aslinguistic asset optimization stage 120 utilizes various linguisticassets.

FIG. 2 is an exemplary diagram depicting a table showing a translationmatch progression through a translation supply chain. Table 200 includesrelative quantity matches and, as those skilled in the art canappreciate, original content may include more or less original segments,along with different matching type ratios, than that shown in FIG. 2.

Table 200 includes column 220, which shows that the language assetoptimization stage parses original content into 1,000 original segments.Column 230 shows that the language asset optimization stage identified400 segments with auto-exact matches, 200 segments with exact matches,and 200 segments with fuzzy matches. As such, 200 segments remain withno match after the language asset optimization stage.

Column 240 shows that the machine translation stage designates 20segments of the 200 segments as placebo segments and generates 180machine translations. In turn, the professional linguist may not reviewthe 400 auto exact matches, but reviews and corrects the 200 segmentswith exact matches, the 200 segments with fuzzy matches, the 180segments with machine translations, and manually translates the 20(placebo) segments with no matches. In one embodiment, the machinetranslation stage designates the placebo segments so linguistic analyzerengine 180 has a baseline set of segments from which to normalize theproductivity of other suggested match types.

FIG. 3 is an exemplary diagram depicting an example of a class set eventdata hierarchy. As discussed herein, a “class set” is a whole space ofevent data and a “subset” of the class set is a child class set. FIG. 3shows that class set event data 300 includes multiple language class setevent data 310, 320, and 330, which are at a major dimension hierarchylevel (e.g., “LANGUAGE”). Within each language class set exists multiplechild class sets. FIG. 3 shows three match type child class sets ofexact match class set event data 340, fuzzy match class set event data350, and machine translation class set event data 360. Although notshown, each of language class set event data 310, 320, and 330 may havea similar hierarchy of child class sets.

Continuing down the hierarchy tree, each match type child class set mayhave child class sets according to segment size. FIG. 3 shows that fuzzymatch class set event data 350 has multiple segment size child classsets, which are small segment class set event data 370, medium segmentclass set event data 380, and large segment class set event data 390. Inone embodiment, small segments may include one to four words, mediumsegments may include five to fourteen words, and large segments mayinclude greater than fourteen words.

FIG. 4 is an exemplary flowchart depicting steps taken by a languageasset optimization stage to generate suggested matches for originalsegments partitioned from original content. The language assetoptimization stage parses new content (original content) into segmentsvia a segmentation process and then compares each new segment againstmultiple historical translation segments to locate a matchingtranslation. The historical translation segments include previoustranslations indexed by contextual metadata keys such as documentversion, brand, project, domain, etc.

Language asset optimization stage processing commences at 400, whereuponthe language asset optimization stage retrieves original content andcontextual metadata from shipment store 110 (step 410). The contextualmetadata may identify, for example, the original content's domain,brand, industry, etc. At step 420, the language asset optimization stageparses the original content into original segments.

At step 430, the language asset optimization stage selects a firstsegment at 430, and performs a global search on historical translationsstored in linguistic asset store 125 to locate a translation of theselected segment (step 440). When the language asset optimization stageidentifies a match meeting a quality threshold, the language assetoptimization stage stores the match in shipment store 110 at step 450.In one embodiment, the language asset optimization stage classifiestranslation memory matches into at least three classes such asauto-exact match, exact match, and fuzzy match as discussed above.

The language asset optimization stage determines whether there are moreoriginal segments to analyze and compare against historical translations(decision 470). If there are more original segments to analyze, decision470 branches to the “Yes” branch, which loops back to select and processthe next segment. This looping continues until there are no moreoriginal segments to evaluate, at which point decision 470 branches tothe “No” branch, whereupon language asset optimization stage processingends at 480.

FIG. 5 is an exemplary flowchart depicting steps taken by a machinetranslation stage to customize a machine translation model and translateoriginal segments into a different language using the customized machinetranslation model. In one embodiment, the machine translation stage useslinguistic and statistical technology to provide the best (mostaccurate) machine translation match per original segment (see FIG. 6 andcorresponding text for further details). In this embodiment, the machinetranslation stage may customize the machine translation model tospecific domains (e.g., subject areas, technologies, etc.) perlanguages.

Machine translation stage processing commences at 500, whereupon themachine translation stage customizes the machine translation model at alanguage level (e.g., Spanish) using linguistic assets corresponding tothe particular language from linguistic asset store 125 (step 510). Atstep 520, the machine translation stage customizes the machinetranslation model at a project domain level using linguistic assetscorresponding to project domain included in linguistic asset store 125,such as a “computing” domain.

In some embodiments, the machine translation stage may use technologiessuch as Statistical Machine Translation (SMT), Rule Based MachineTranslation (RBMT), or Hybrid Machine Translation (HMT) to optimize themachine translation engine. SMT uses a translation database thatincludes final human fluent translations previously quantified by humanprofessional linguists. RBMT establishes special lexicon, morphological,grammatical, syntactic and custom rules tailored to the language undertranslation. HMT uses a combination of learning memories and rules.

In another embodiment, the machine translation stage strives to optimizethe quality of the machine translation matches per domain per languageby using custom learning linguistic assets (quality translationmemories) per domain per language, using custom linguistic rules perdomain per language, and/or pre-existing knowledge of linguisticpatterns. In this embodiment when perfect linguistic assets and/orlinguistic rules per domain per language are utilized, the machinetranslation stage should provide 100% human-fluent domain specifictranslations. However, linguistic assets, linguistic rules andcustomization tasks all contribute errors or “linguistic noise” thatprevents the machine translation stage from delivering perfect humanfluent translations (discussed below).

At step 530, the machine translation stage retrieves original segmentsand best matches per shipment from linguistic asset store 125, andperforms machine translations on the retrieved segments at step 540. Atstep 550, the machine translation stage stores the machine translationmatches in shipment store 110. Machine translation processing ends at560.

FIG. 6 is an exemplary flowchart depicting steps taken by a post-editingstage that provides suggested translation matches to a professionallinguist and stores final translations determined by the professionallinguist. During the post-editing stage, the professional linguistevaluates suggested matches to determine a best translation match. Inone embodiment, the professional linguist uses a computer-aidedtranslation editor to cycle through each original segment to produce afinal human-fluent translation for each original segment. In thisembodiment, the computer-aided translation editor aids the professionallinguist by displaying the best suggested translation for each newsegment, such as displaying an exact match over a fuzzy match. In turn,the professional linguist may accept the best match or reject allmatches and correct/edit the final translation, which the post-editingstage stores as a final human-fluent translation.

Post-editing stage processing commences at 600, whereupon thepost-editing stage retrieves the shipment's original segments (step 610)and the suggested matches (step 620) from shipment store 110. Theprofessional linguist selects the first segment at 630 and thecomputer-aided translation editor identifies a best match case at step640. For example, the computer-aid translation editor selects an exactmatch over a fuzzy match as the best match.

At step 650, the post-editing stage receives a user action of acceptingthe best match, accepting one of the other matches, or rejecting all thematches. The post-editing stage, at step 655, receives corrections fromthe professional linguist and corrects the selected match. In turn, thepost-editing stage stores the final translation in shipment store 110 toreturn to the shipment owner at step 660. In addition, the post-editingstage stores the final translated content in linguist asset store 125 toimprove future translations, and stores editing event data into eventstore 175 for subsequent analysis (see FIG. 7 and corresponding text forfurther details).

The post-editing stage determines whether there are more segments toevaluate (decision 670). If there are more segments to evaluate,decision 670 branches to the “Yes” branch, which loops back to selectand process the next segment. This looping continues until there are nomore segments to evaluate, at which point decision 670 branches to the“No” branch, whereupon processing ends at 680.

In another embodiment, a localization service provider (LSP) uses thetotal time for post-editing and the total number of words post-edited tosample productivity over a set of post-editing events. In thisembodiment, the LSP collects editing event data into log data and thenprovides the log data to a backend business analytical process, whichassesses the productivity gained from a professional linguist selectingspecific match types. In this embodiment, the log data may include thenumber of words per segment, time to complete each translation, numberof keystrokes entered per translation, type of match accepted, and thebest match type proposed per segment (e.g., exact match, fuzzy match, ormachine translation).

FIG. 7 an exemplary flowchart depicting steps taken by a linguisticanalyzer engine to analyze translation log data and determine linguisticnoise in a translation supply chain.

Linguistic analyzer engine processing commences at 700, whereupon thelinguistic analyzer engine retrieves periodic event data, such as sampleevent data over 12 months, from editing event store 175 (step 705). Thelinguistic analyzer engine sorts/aggregates the periodic event data intochild class sets at step 710 (see FIG. 3 and corresponding text forfurther details). In one embodiment, the linguistic analyzer engine“trims” each child class set by removing outlier “slow” events andoutlier “fast” events (e.g., at 5 and 95 percentiles of the child classsets).

The linguistic analyzer engine selects a major dimension at step 715,such as a language, and computes a placebo baseline productivity value(BPV) at step 720 for the placebo baseline child class set using aformula such as:

${{NP}_{—}{Productivity}_{s}} = {\bigcup\limits_{S}\left( {\sum\limits_{i = 1}^{n}\;{{NP}_{—}{WORDS}_{S}\text{/}{\sum\limits_{i = 1}^{n}\;{{NP}_{—}{TIME}_{S}}}}} \right)}$where S=each segment size [small, medium, large], and n=number of eventswithin the set [S].

At step, 725, the linguistic analyzer engine computes a matchproductivity value (MPV) for each child class set (child class matchproductivity value) using on formulas such as:

${{EM}_{—}{Productivity}_{S}^{M}} = {\bigcup\limits_{S}^{M}\left( {\sum\limits_{i = 1}^{n}\;{{EM}_{—}{WORDS}_{S}^{M}\text{/}{\sum\limits_{i = 1}^{n}\;{{EM}_{—}{TIME}_{S}^{M}}}}} \right)}$${{FM}_{—}{Productivity}_{S}^{M}} = {\bigcup\limits_{S}^{M}\left( {\sum\limits_{i = 1}^{n}\;{{FM}_{—}{WORDS}_{S}^{M}\text{/}{\sum\limits_{i = 1}^{n}\;{\left( {FM} \right)_{—}{TIME}_{S}^{M}}}}} \right)}$${{MT}_{—}{Productivity}_{S}^{M}} = {\bigcup\limits_{S}^{M}\left( {\sum\limits_{i = 1}^{n}\;{{MT}_{—}{WORDS}_{S}^{M}\text{/}{\sum\limits_{i = 1}^{n}\;{{MT}_{—}{TIME}_{S}^{M}}}}} \right)}$where M=each MAJORKEY per major dimension (first child dimension ofparent dimension), m=number of events per MAJORKEY per segment scopesize, and n=number of events within the set [M,S].

In turn, the linguistic analyzer engine computes a post-editing factorvalue for each child class set (child class factor value) at step 730.In one embodiment, PEFVs are computed for specific class sets usingformulas such as:

${{EM}_{—}{Factor}_{S}^{M}} = {\bigcup\limits_{S}^{M}{{NP}_{—}{Productivity}_{S}\text{/}{EM}_{—}{Productivity}_{S}^{M}}}$${{FM}_{—}{Factor}_{S}^{M}} = {\bigcup\limits_{S}^{M}{{NP}_{—}{Productivity}_{S}\text{/}{FM}_{—}{Productivity}_{S}^{M}}}$${{MT}_{—}{Factor}_{S}^{M}} = {\bigcup\limits_{S}^{M}{{NP}_{—}{Productivity}_{S}\text{/}{MT}_{—}{Productivity}_{S}^{M}}}$where m=number of events per MAJORKEY per segment scope size. In oneembodiment, the PEFV is a measurement of mechanical effort by theprofessional linguist during post-editing needed to eliminate noise(error) in the selected matches. In another embodiment, the PEFV is ameasurement of “normalized professional linguist productivity” duringpost-editing, such as translations performed with the aid of a specificmatch types or other aids.

At step 735, the linguistic analyzer engine computes the total words(TWct) in each child class set and computes the total accepted words(AWct) in each child class set (step 740) using formulas such as:

${{EM}_{—}{Total}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;{{EM}_{—}{Words}_{S}^{M}}}}$${{FM}_{—}{Total}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;{{FM}_{—}{Words}_{S}^{M}c}}}$${{MT}_{—}{Total}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;{{MT}_{—}{Words}_{S}^{M}}}}$${{EM}_{—}{Accept}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;\left( {{{EM}_{—}{Words}}\bigcap\limits_{S}^{M}\left\lbrack {{Accept} = {Yes}} \right\rbrack} \right)}}$${{FM}_{—}{Accept}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;\left( {{{FM}_{—}{Words}}\bigcap\limits_{S}^{M}\left\lbrack {{Accept} = {Yes}} \right\rbrack} \right)}}$${{MT}_{—}{Accept}_{—}{Words}_{S}^{M}} = {\bigcup\limits_{S}^{M}{\sum\limits_{i = 1}^{n}\;\left( {{{MT}_{—}{Words}}\bigcap\limits_{S}^{M}\left\lbrack {{Accept} = {Yes}} \right\rbrack} \right)}}$

Using the AWct and TWct, the linguistic analyzer engine computes apost-editing leverage value (PELV) for each class set (PELV=AWct/TWct)at step 745 (child class leverage value) using formulas such as:EM_Leverage_(S) ^(M)=EM_Accept_Words_(S) ^(M)/EM_Total_Words_(S) ^(M)FM_Leverage_(S) ^(M)=FM_Accept_Words_(S) ^(M)/FM_Total_Words_(S) ^(M)MT_Leverage_(S) ^(M)=MT_Accept_Words_(S) ^(M)/MT_Total_Words_(S) ^(M)

In one embodiment, the PELV is a measurement of acceptance of matches bythe professional linguist during post-editing. In this embodiment, thePELV is a measurement such that 1.0 indicates that the professionallinguist selected 100% of the suggested translations provided for theaided class data set. For example, if a human translator chooses 50% ofthe suggested translations (measured in total words accepted), then avalue of “0.5” statistically measures the quality of matches provided tothe professional linguist based on human cognitive judgment.

At step 750, the linguistic analyzer engine computes linguistic markers(LM) using the PELV and PEFV along with other metrics using formulassuch as:

${LinguisticMarker}_{S}^{M} = \begin{bmatrix}\left\lbrack {{{EM}_{—}{Leverage}_{S}^{M}},{{EM}_{—}{Factor}_{S}^{M}}} \right\rbrack \\\left\lbrack {{{FM}_{—}{Leverage}_{S}^{M}},{{FM}_{—}{Factor}_{S}^{M}}} \right\rbrack \\\left\lbrack {{{MT}_{—}{Leverage}_{S}^{M}},{{MT}_{—}{Factor}_{S}^{M}}} \right\rbrack\end{bmatrix}$

In one embodiment, the linguistic markers are coordinates for graphingthe professional linguist's acceptance of matches relative to theprofessional linguist's productivity (see FIG. 8 and corresponding textfor further details).

In another embodiment, the closer the coordinates of a linguistic markerare to coordinates of a Perfect Linguistic Marker (PLM) [1.0,0.0], theless noise is present in the translation supply chain. As such, thelinguistic analyzer engine may identify and provide analytical feedbackusing the linguistic markers whose coordinates are furthest away fromthe PLM. In addition, the linguistic analyzer engine may analyze a setof linguistic markers across the set of match types per segment scopesize. By analyzing the relationship between these sets of linguisticmarkers, the linguistic analyzer engine identifies patterns such as:

-   -   small PE:Factor<medium PE:Factor<large PE:Factor; and    -   exact match PE:Factor<fuzzy match PE:Factor<machine translation        PE:Factor.

The linguistic analyzer engine may also determine that the post-editingfactor value increases if the post-editing leverage value is too high.In other words, a professional linguist's productivity may suffer whenthe professional linguist selects too many bad matches for which toedit.

In yet another embodiment, the linguistic analyzer engine may buildstatistical models to assess and identify patterns between exact matchlinguistic markers, fuzzy match linguistic markers and machinetranslation linguistic markers. This analysis aids an operational teamto establish benchmarks and boundary exception conditions to pinpointindicators of high levels of noise in the operational assets/resourcesand/or components.

In yet another embodiment, the linguistic analyzer engine may identify asubset of shipments, from a large set of shipments, with the worstlinguistic markers and analyze the corresponding document levels toidentify the subset of documents with the worst linguistic markers. Inthis embodiment, the linguistic analyzer engine may traverse into themultiple segments with the worst linguistic marker coordinates andquickly identify/qualify the worst match types within the MAJORKEY andhelp make investment decisions on whether to correct the issues.

At step 755, the linguistic analyzer engine computes a linguistic vectorper class (child class linguistic vector) set using formulas such as:

${{}_{}^{}{}_{}^{}} = {\sqrt{{{}_{}^{}{}_{}^{}} + \left( {1 - {\,_{T}{Leverage}}} \right)^{2}}}_{S}^{M}$${LinguisticVector}_{S}^{M} = \begin{bmatrix}\left\lbrack {{EM}_{—}{Vector}_{S}^{M}} \right\rbrack \\\left\lbrack {{FM}_{—}{Vector}_{S}^{M}} \right\rbrack \\\left\lbrack {{MT}_{—}{Vector}_{S}^{M}} \right\rbrack\end{bmatrix}$where T=Match Type [EM, FM, MT]. In one embodiment, the linguisticvector is a measurement of distance from a linguistic marker to thePerfect Linguist Marker (see PLM 835). In this embodiment, thelinguistic vector is a singular composite measurement of class set noise(child class set linguistic noise value) such that the linguistic vectorreflects the distance from PLM 835 to a linguistic marker, or the amountof work needed to achieve perfect matches. In this embodiment, thelinguistic vector is a composite of post-editing factor noise andpost-editing leverage noise. The post-editing factor noise representsthe mechanical effort to remove linguistic noise contributed from thevarious translation supply chain stages, and the post-editing leveragenoise is a measurement of the cognitive assessment of the best match.For example, if the post-editing leverage is 0.5, then 50% of thesuggested matches are not accurate enough to correct (see FIG. 8 andcorresponding text for further details).

At step 765, the linguistic analyzer engine computes a weighted sumlinguistic noise for each MAJORKEY (major linguistic noise) using aformulas such as:

${LinguisticNoise}^{M} = {\sum\limits_{S}^{T}\left( {{{}_{}^{}{}_{}^{}}{X\left( {{{}_{}^{}{}_{}^{}}{WORDS}_{S}^{M}\text{/}{Total}_{—}{WORDS}^{M}} \right)}} \right)}$where T=Match Type [EM, FM, MT].

In one embodiment, the major linguistic noise is the sum measurement ofall class set noise (linguistic vectors) weighted by the respectiveclass set percentage of words within a MAJORKEY. The major linguisticnoise value represents the distance from a set of perfect matches(Linguistic Vector) weighted by the volume of words within a pluralityof matches. If the linguistic vector (class set noise) of all class setsreflected a measurement of zero, then the major linguistic noise wouldreflect zero linguistic noise reflecting 100% perfect matches within theMAJORKEY sampled event data 170 within event store 175.

The linguistic analyzer engine then sums the weighted sum majorlinguistic nose across all major dimensions to compute a performanceefficiency (e.g., translation supply chain linguistic noise value) atstep 770 using a formula such as:

${SupplyChainNoise} = {\sum\limits_{i = 1}^{m}\;\left( {{LinguisticNoise}^{m}{X\left( {{Total}_{—}{WORDS}^{m}\text{/}{\sum\limits_{i = 1}^{n}\;{WORDS}^{M}}} \right)}} \right)}$where M=MAJORKEY (default=LOGNAME, shipment), m=each MAJORKEY, andn=total number of all events (MAJORKEY).

The linguistic analyzer engine evaluates the efficiency of a translationsupply chain (step 775) based on the translation supply chain noise(from step 770) and processing ends at 780. In one embodiment, thetranslation supply chain noise evaluation involves the linguisticanalyzer engine and an operational team providing analytical feedback tocomponents and an operational assets/resources in the translation supplychain. In this embodiment, the linguistic analyzer engine evaluates howchanges in operational assets/resources and components affect patternsin the translation supply chain. For example, the operational team mayquantify changes in the supply chain noise as various machinetranslation service providers are added/deleted into the translationsupply chain, or as various operational assets/resource areadded/deleted or different methods are applied in the management of theoperational assets/resources.

In another embodiment, by tracking the translation supply chainlinguistic noise across all MAJORKEY instances over time, the linguisticanalyzer engine and operational team may build a simple overall processbehavior view such that the linguistic analyzer engine and operationalteam identifies boundary exception conditions and implements correctiveaction plans to minimize future occurrences of those exceptions. Inaddition, the linguistic analyzer engine and operational team mayprovide analytical feedback to the machine translation stage forinvestment decisions and selecting specific noise variables for which toimprove.

FIG. 8 is an exemplary diagram depicting a two-dimensional graphicalrepresentation of a multi-dimensional linguistic noise coordinatesystem. Graph 800 plots linguistic markers according to post-editingleverage values (x-axis) and post-editing factor values (y-axis).

Graph 800 includes nine linguistic markers categorized by match type andmatch size. Circles represent large segment sizes, triangles representmedium segment sizes, and squares represent the small segment sizes. Thethick-lined objects represent machine translations (MM), thedashed-lined objects represent fuzzy matches (FM), and the thin-linedobjects represent exact matches (EM).

As such, L-MT 810 represents large segment machine translations, M-MT812 represents medium segment machine translations, and S-MT 814represents small segment machine translations. Likewise, L-FM 820represents large segment fuzzy matches, M-FM 822 represents mediumsegment fuzzy matches, and S-FM 824 represents small segment fuzzymatches. Finally, L-EM 830 represents large segment exact matches, M-EM832 represents medium segment exact matches, and S-EM 834 representssmall segment exact matches. As can be seen, the exact match linguisticmarkers are closest to PLM 935 because they are typically bettertranslations when compared against fuzzy matches or machinetranslations.

Graph 800 is one embodiment of a multidimensional linguistic noisecoordinate system showing an exemplary set of linguistic markers andtheir corresponding linguistic vector(s) associated with class set eventdata 300. Linguistic vector 840, which corresponds to marker M-EM 832,reflects the linguistic noise contributed by the class set ofmedium-exact match events (M-EM 832) in event store 175. Post-editingfactor noise 850 represents one dimension of the vector (class setnoise) 840 based on measuring the mechanical effort used to eliminatethe linguistic noise within the matches in the class set M-EM 832. Forexample, if all matches associated with the events of marker M-EM 832were all perfect, then post-editing factor noise 850 would measure 0.0,reflecting zero mechanical effort to clean all the matches. Likewise,post-editing leverage noise 860 represents a second dimension of thevector (class set noise) 840 based on measuring the human cognitiveassessments (decisions) across all the matches used within class setM-EM 832. For example, if all the matches associated with the events ofmarker M-EM 832 were all perfect, then the post-editing leverage noisedimension 860 would measure 1.0, reflecting 100% acceptance of allmatches. Each marker (L-MT 810, M-MT 812, S-MT 814, L-FM 820, M-FM 822,S-FM 824, L-EM 830, M-EM 832, and S-EM 834) and their correspondingvectors from Perfect Linguistic Marker 835 point (1.0, 0.0) representstheir respective contribution of linguistic noise, per class set, to thetranslation supply chain linguistic noise (computed in step 770).

FIG. 9 illustrates information handling system 900, which is asimplified example of a computer system capable of performing thecomputing operations described herein. Information handling system 900includes one or more processors 910 coupled to processor interface bus912. Processor interface bus 912 connects processors 910 to Northbridge915, which is also known as the Memory Controller Hub (MCH). Northbridge915 connects to system memory 920 and provides a means for processor(s)910 to access the system memory. Graphics controller 925 also connectsto Northbridge 915. In one embodiment, PCI Express bus 918 connectsNorthbridge 915 to graphics controller 925. Graphics controller 925connects to display device 930, such as a computer monitor.

Northbridge 915 and Southbridge 935 connect to each other using bus 919.In one embodiment, the bus is a Direct Media Interface (DMI) bus thattransfers data at high speeds in each direction between Northbridge 915and Southbridge 935. In another embodiment, a Peripheral ComponentInterconnect (PCI) bus connects the Northbridge and the Southbridge.Southbridge 935, also known as the I/O Controller Hub (ICH) is a chipthat generally implements capabilities that operate at slower speedsthan the capabilities provided by the Northbridge. Southbridge 935typically provides various busses used to connect various components.These busses include, for example, PCI and PCI Express busses, an ISAbus, a System Management Bus (SMBus or SMB), and/or a Low Pin Count(LPC) bus. The LPC bus often connects low-bandwidth devices, such asboot ROM 996 and “legacy” I/O devices (using a “super I/O” chip). The“legacy” I/O devices (998) can include, for example, serial and parallelports, keyboard, mouse, and/or a floppy disk controller. The LPC busalso connects Southbridge 935 to Trusted Platform Module (TPM) 995.Other components often included in Southbridge 935 include a DirectMemory Access (DMA) controller, a Programmable Interrupt Controller(PIC), and a storage device controller, which connects Southbridge 935to nonvolatile storage device 985, such as a hard disk drive, using bus984.

ExpressCard 955 is a slot that connects hot-pluggable devices to theinformation handling system. ExpressCard 955 supports both PCI Expressand USB connectivity as it connects to Southbridge 935 using both theUniversal Serial Bus (USB) the PCI Express bus. Southbridge 935 includesUSB Controller 940 that provides USB connectivity to devices thatconnect to the USB. These devices include webcam (camera) 950, infrared(IR) receiver 948, keyboard and trackpad 944, and Bluetooth device 946,which provides for wireless personal area networks (PANs). USBController 940 also provides USB connectivity to other miscellaneous USBconnected devices 942, such as a mouse, removable nonvolatile storagedevice 945, modems, network cards, ISDN connectors, fax, printers, USBhubs, and many other types of USB connected devices. While removablenonvolatile storage device 945 is shown as a USB-connected device,removable nonvolatile storage device 945 could be connected using adifferent interface, such as a Firewire interface, etcetera.

Wireless Local Area Network (LAN) device 975 connects to Southbridge 935via the PCI or PCI Express bus 972. LAN device 975 typically implementsone of the IEEE 802.11 standards of over-the-air modulation techniquesthat all use the same protocol to communicate wirelessly betweeninformation handling system 900 and another computer system or device.Optical storage device 990 connects to Southbridge 935 using Serial ATA(SATA) bus 988. Serial ATA adapters and devices communicate over ahigh-speed serial link. The Serial ATA bus also connects Southbridge 935to other forms of storage devices, such as hard disk drives. Audiocircuitry 960, such as a sound card, connects to Southbridge 935 via bus958. Audio circuitry 960 also provides functionality such as audioline-in and optical digital audio in port 962, optical digital outputand headphone jack 964, internal speakers 966, and internal microphone968. Ethernet controller 970 connects to Southbridge 935 using a bus,such as the PCI or PCI Express bus. Ethernet controller 970 connectsinformation handling system 900 to a computer network, such as a LocalArea Network (LAN), the Internet, and other public and private computernetworks.

While FIG. 9 shows one information handling system, an informationhandling system may take many forms. For example, an informationhandling system may take the form of a desktop, server, portable,laptop, notebook, or other form factor computer or data processingsystem. In addition, an information handling system may take other formfactors such as a personal digital assistant (PDA), a gaming device, ATMmachine, a portable telephone device, a communication device or otherdevices that include a processor and memory.

The Trusted Platform Module (TPM 995) shown in FIG. 9 and describedherein to provide security functions is but one example of a hardwaresecurity module (HSM). Therefore, the TPM described and claimed hereinincludes any type of HSM including, but not limited to, hardwaresecurity devices that conform to the Trusted Computing Groups (TCG)standard, and entitled “Trusted Platform Module (TPM) SpecificationVersion 1.2.” The TPM is a hardware security subsystem that may beincorporated into any number of information handling systems, such asthose outlined in FIG. 10.

FIG. 10 provides an extension of the information handling systemenvironment shown in FIG. 9 to illustrate that the methods describedherein can be performed on a wide variety of information handlingsystems that operate in a networked environment. Types of informationhandling systems range from small handheld devices, such as handheldcomputer/mobile telephone 1010 to large mainframe systems, such asmainframe computer 1070. Examples of handheld computer 1010 includepersonal digital assistants (PDAs), personal entertainment devices, suchas MP3 players, portable televisions, and compact disc players. Otherexamples of information handling systems include pen, or tablet,computer 1020, laptop, or notebook, computer 1030, workstation 1040,personal computer system 1050, and server 1060. Other types ofinformation handling systems that are not individually shown in FIG. 10are represented by information handling system 1080. As shown, thevarious information handling systems can be networked together usingcomputer network 1000. Types of computer network that can be used tointerconnect the various information handling systems include Local AreaNetworks (LANs), Wireless Local Area Networks (WLANs), the Internet, thePublic Switched Telephone Network (PSTN), other wireless networks, andany other network topology that can be used to interconnect theinformation handling systems. Many of the information handling systemsinclude nonvolatile data stores, such as hard drives and/or nonvolatilememory. Some of the information handling systems shown in FIG. 10depicts separate nonvolatile data stores (server 1060 utilizesnonvolatile data store 1065, mainframe computer 1070 utilizesnonvolatile data store 1075, and information handling system 1080utilizes nonvolatile data store 1085). The nonvolatile data store can bea component that is external to the various information handling systemsor can be internal to one of the information handling systems. Inaddition, removable nonvolatile storage device 945 can be shared amongtwo or more information handling systems using various techniques, suchas connecting the removable nonvolatile storage device 945 to a USB portor other connector of the information handling systems.

While particular embodiments of the present disclosure have been shownand described, it will be obvious to those skilled in the art that,based upon the teachings herein, that changes and modifications may bemade without departing from this disclosure and its broader aspects.Therefore, the appended claims are to encompass within their scope allsuch changes and modifications as are within the true spirit and scopeof this disclosure. Furthermore, it is to be understood that thedisclosure is solely defined by the appended claims. It will beunderstood by those with skill in the art that if a specific number ofan introduced claim element is intended, such intent will be explicitlyrecited in the claim, and in the absence of such recitation no suchlimitation is present. For non-limiting example, as an aid tounderstanding, the following appended claims contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimelements. However, the use of such phrases should not be construed toimply that the introduction of a claim element by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim element to disclosures containing only one suchelement, even when the same claim includes the introductory phrases “oneor more” or “at least one” and indefinite articles such as “a” or “an”;the same holds true for the use in the claims of definite articles.

The invention claimed is:
 1. A method implemented by an informationhandling system that includes a memory, instructions stored in thememory, and a processor, the method comprising: computing, by theprocessor using a first set of instructions stored in the memory, aleverage value of a language translation supply chain, wherein theleverage value corresponds to an amount of suggested translations, froma plurality of suggested translations, that are accepted by a user thatresults in a set of accepted translations; computing, by the processorusing a second set of instructions stored in the memory, a factor valueof the language translation supply chain, wherein the factor valueindicates a productivity of the user to convert the set of acceptedtranslation into a set of final translations; determining, by theprocessor, a performance efficiency of the language translation supplychain based upon the leverage value and the factor value; and evaluatingthe language translation supply chain based upon the performanceefficiency.
 2. The method of claim 1 wherein the plurality of suggestedtranslations correspond to a plurality of child class sets, the methodfurther comprising: computing a child class set linguistic noise valuefor each of the plurality of child class sets using a correspondingchild class leverage value and a child class factor value, resulting ina plurality of child class set linguistic noise values; combining theplurality of child class set linguistic noise values, resulting in atranslation supply chain linguistic noise value; and utilizing thetranslation supply chain linguistic noise value during the performanceefficiency determination of the language translation supply chain. 3.The method of claim 2 further comprising: selecting one of the pluralityof child class sets; identifying a total amount of words included in theplurality of suggested translations that each correspond to the selectedchild class set; identifying a total amount of accepted words includedin the set of accepted translations that each correspond to the selectedchild class set; and computing the child class leverage value of theselected child class set using the identified total amount of words andthe total amount of accepted words.
 4. The method of claim 2 whereineach of the plurality of suggested translations corresponds to one of aplurality of original segments written in a first language and one of aplurality of corresponding translation segments written in a secondlanguage, the method further comprising: designating a set of theplurality of original segments as a set of placebo segments, wherein thetranslation supply chain prevents generation of a suggested translationfor each of the segments included in the set of placebo segments;computing a placebo baseline productivity value based upon the set ofplacebo segments that do not correspond to one of the plurality ofsuggested translations; selecting one of the plurality of child classsets; computing a child class match productivity value of the selectedchild class based upon the set of accepted translations that eachcorrespond to the selected child class set; and computing the childclass factor value of the selected child class set using the placebobaseline productivity value and the child class match productivityvalue.
 5. The method of claim 4 further comprising: tracking one or morefirst user actions of the user to generate one or more finaltranslations corresponding to the set of placebo segments, the one ormore first user actions utilized to compute the placebo baselineproductivity value; and tracking one or more second user actions of theuser to convert one or more of the set of accepted translations to oneor more of the set of final translations, the one or more second useractions utilized to compute the child class match productivity value. 6.The method of claim 2 wherein at least one of the plurality of childclass sets corresponds to a match type selected from the groupconsisting of an auto exact match type, an exact match type, a fuzzymatch type, and a machine translation match type.
 7. The method of claim1 wherein the language translation supply chain comprises one or morefirst stages and a second stage that is subsequent to the one or morefirst stages, the method further comprising: generating the plurality ofsuggested translations by the one or more first stages; converting, bythe second stage, the set of accepted translations into the set of finaltranslations; and adjusting the one or more first stages in response tothe evaluating to improve the performance efficiency of the languagetranslation supply chain.
 8. An information handling system comprising:one or more processors; a memory coupled to at least one of theprocessors; a set of computer program instructions stored in the memoryand executed by at least one of the processors in order to performactions of: computing a leverage value of a language translation supplychain, wherein the leverage value corresponds to an amount of suggestedtranslations, from a plurality of suggested translations, that areaccepted by a user that results in a set of accepted translations;computing a factor value of the language translation supply chain,wherein the factor value indicates a productivity of the user to convertthe set of accepted translation into a set of final translations;determining a performance efficiency of the language translation supplychain based upon the leverage value and the factor value; and evaluatingthe language translation supply chain based upon the performanceefficiency.
 9. The information handling system of claim 8 wherein theplurality of suggested translations correspond to a plurality of childclass sets, and wherein the processors perform additional actionscomprising: computing a child class set linguistic noise value for eachof the plurality of child class sets using a corresponding child classleverage value and a child class factor value, resulting in a pluralityof child class set linguistic noise values; combining the plurality ofchild class set linguistic noise values, resulting in a translationsupply chain linguistic noise value; and utilizing the translationsupply chain linguistic noise value during the performance efficiencydetermination of the language translation supply chain.
 10. Theinformation handling system of claim 9 wherein the processors performadditional actions comprising: selecting one of the plurality of childclass sets; identifying a total amount of words included in theplurality of suggested translations that each correspond to the selectedchild class set; identifying a total amount of accepted words includedin the set of accepted translations that each correspond to the selectedchild class set; and computing the child class leverage value of theselected child class set using the identified total amount of words andthe total amount of accepted words.
 11. The information handling systemof claim 9 wherein each of the plurality of suggested translationscorresponds to one of a plurality of original segments written in afirst language and one of a plurality of corresponding translationsegments written in a second language, and wherein the processorsperform additional actions comprising: designating a set of theplurality of original segments as a set of placebo segments, wherein thetranslation supply chain prevents generation of a suggested translationfor each of the segments included in the set of placebo segments;computing a placebo baseline productivity value based upon the set ofplacebo segments that do not correspond to one of the plurality ofsuggested translations; selecting one of the plurality of child classsets; computing a child class match productivity value of the selectedchild class based upon the set of accepted translations that eachcorrespond to the selected child class set; and computing the childclass factor value of the selected child class set using the placebobaseline productivity value and the child class match productivityvalue.
 12. The information handling system of claim 11 wherein theprocessors perform additional actions comprising: tracking one or morefirst user actions of the user to generate one or more finaltranslations corresponding to the set of placebo segments, the one ormore first user actions utilized to compute the placebo baselineproductivity value; and tracking one or more second user actions of theuser to convert one or more of the set of accepted translations to oneor more of the set of final translations, the one or more second useractions utilized to compute the child class match productivity value.13. The information handling system of claim 9 wherein at least one ofthe plurality of child class sets corresponds to a match type selectedfrom the group consisting of an auto exact match type, an exact matchtype, a fuzzy match type, and a machine translation match type.
 14. Theinformation handling system of claim 8 wherein the language translationsupply chain comprises one or more first stages and a second stage thatis subsequent to the one or more first stages, and wherein theprocessors perform additional actions comprising: generating theplurality of suggested translations by the one or more first stages;converting, by the second stage, the set of accepted translations intothe set of final translations; and adjusting the one or more firststages in response to the evaluating to improve the performanceefficiency of the language translation supply chain.
 15. A computerprogram product comprising a computer readable storage medium havingprogram instructions stored therewith, the program instructionsexecutable by a processor to cause an information handling system toperform a method comprising: computing a leverage value of a languagetranslation supply chain, wherein the leverage value corresponds to anamount of suggested translations, from a plurality of suggestedtranslations, that are accepted by a user that results in a set ofaccepted translations; computing a factor value of the languagetranslation supply chain, wherein the factor value indicates aproductivity of the user to convert the set of accepted translation intoa set of final translations; determining a performance efficiency of thelanguage translation supply chain based upon the leverage value and thefactor value; and evaluating the language translation supply chain basedupon the performance efficiency.
 16. The computer program product ofclaim 15 wherein the plurality of suggested translations correspond to aplurality of child class sets, and wherein the information handlingsystem performs further actions comprising: computing a child class setlinguistic noise value for each of the plurality of child class setsusing a corresponding child class leverage value and a child classfactor value, resulting in a plurality of child class set linguisticnoise values; combining the plurality of child class set linguisticnoise values, resulting in a translation supply chain linguistic noisevalue; and utilizing the translation supply chain linguistic noise valueduring the performance efficiency determination of the languagetranslation supply chain.
 17. The computer program product of claim 16wherein the information handling system performs further actionscomprising: selecting one of the plurality of child class sets;identifying a total amount of words included in the plurality ofsuggested translations that each correspond to the selected child classset; identifying a total amount of accepted words included in the set ofaccepted translations that each correspond to the selected child classset; and computing the child class leverage value of the selected childclass set using the identified total amount of words and the totalamount of accepted words.
 18. The computer program product of claim 16wherein each of the plurality of suggested translations corresponds toone of a plurality of original segments written in a first language andone of a plurality of corresponding translation segments written in asecond language, and wherein the information handling system performsfurther actions comprising: designating a set of the plurality oforiginal segments as a set of placebo segments, wherein the translationsupply chain prevents generation of a suggested translation for each ofthe segments included in the set of placebo segments; computing aplacebo baseline productivity value based upon the set of placebosegments that do not correspond to one of the plurality of suggestedtranslations; selecting one of the plurality of child class sets;computing a child class match productivity value of the selected childclass based upon the set of accepted translations that each correspondto the selected child class set; and computing the child class factorvalue of the selected child class set using the placebo baselineproductivity value and the child class match productivity value.
 19. Thecomputer program product of claim 18 wherein the information handlingsystem performs further actions comprising: tracking one or more firstuser actions of the user to generate one or more final translationscorresponding to the set of placebo segments, the one or more first useractions utilized to compute the placebo baseline productivity value; andtracking one or more second user actions of the user to convert one ormore of the set of accepted translations to one or more of the set offinal translations, the one or more second user actions utilized tocompute the child class match productivity value.
 20. The computerprogram product of claim 15 wherein the language translation supplychain comprises one or more first stages and a second stage that issubsequent to the one or more first stages, and wherein the informationhandling system performs further actions comprising: generating theplurality of suggested translations by the one or more first stages;converting, by the second stage, the set of accepted translations intothe set of final translations; and adjusting the one or more firststages in response to the evaluating to improve the performanceefficiency of the language translation supply chain.